Conventionally, a technique for removing noise components from a digital image on which noise components that are different from signal components are superposed has been studied. The characteristics of noise to be removed are diverse depending on their generation factors, and noise removal methods suited to those characteristics have been proposed. For example, when an image input device such as a digital camera, image scanner, or the like is assumed, noise components are roughly categorized into noise which depends on the input device characteristics of a solid-state image sensing element or the like and input conditions such as an image sensing mode, scene, or the like, and has already been superposed on a photoelectrically converted analog original signal, and noise which is superposed via various digital signal processes after the analog signal is converted into a digital signal via an A/D converter.
As an example of the former, impulse noise that generates an isolated value to have no correlation with surrounding image signal values, noise resulting from the dark current of the solid-state image sensing element, and the like are known. As an example of the latter, noise components are amplified simultaneously with signal components when a specific density, color, and the like are emphasized in various correction processes such as gamma correction, gain correction for improving the sensitivity, and the like, thus increasing the noise level. As an example of deterioration due to a digital signal process, since an encoding process using a JPEG algorithm extracts a plurality of blocks from a two-dimensional (2D) image, and executes orthogonal transformation and quantization for respective blocks, a decoded image suffers block distortion that generates steps at the boundaries of blocks.
In addition to various kinds of noise mentioned above, a factor that especially impairs the image quality is noise (to be referred to as “low-frequency noise” hereinafter) which is generated in a low-frequency range and is conspicuously observed in an image sensed by a digital camera or the like. This low-frequency noise of ten results from the sensitivity of a CCD or CMOS sensor as a solid-state image sensing element. In an image sensing scene such as a dark scene with a low signal level, a shadowy scene, or the like, low-frequency noise is often emphasized due to gain correction that raises signal components irrespective of poor S/N ratio. Furthermore, the element sensitivity of the solid-state image sensing element depends on its chip area. Hence, in a digital camera which has a large number of pixels within a small area, the amount of light per unit pixel consequently decreases, and the sensitivity lowers, thus producing noise. Such low-frequency noise is often visually recognized as pseudo mottled texture across several to ten-odd pixels on a flat portion such as a sheet of blue sky or the like. Some digital cameras often produce false colors.
As a conventionally proposed noise removal method, a method using a median filter (to be abbreviated as “MF” hereinafter) which extracts a pixel value which assumes a median from those of a pixel of interest and its surrounding pixels, and replaces the pixel value of interest by the extracted value is prevalent.
Also, as a noise removal method effective for impulse noise, block distortion mentioned above, and the like, a method using a low-pass filter (to be abbreviated as “LPF” hereinafter) which calculates the weighted mean using the pixel values of a pixel of interest and its surrounding pixels, and replaces the pixel value of interest by the calculated weighted mean is used. Furthermore, as a noise removal method effective for low-frequency noise, a method of replacing a pixel value of interest by a pixel value which is probabilistically selected from those around the pixel of interest (to be referred to as a “noise distribution method” hereinafter) has been proposed.
As described above, noise components superposed on an image are influenced by various causes. For example, a digital camera suffers multiple causes, i.e., noise superposed on an analog original signal depending on, e.g., the input device characteristics of a solid-state image sensing element or the like is further amplified by various digital image processes executed after the analog original signal is converted into a digital signal via an A/D converter.
In digital image processes executed by a digital camera, typical processes which are involved in noise generation and amplification include white balance correction, gain correction for improving the sensitivity, saturation correction which is executed for specific colors to simulate memory colors, and the like.
The white balance correction corrects a phenomenon that an originally white image does not appear white due to total color unbalance which is caused since the amounts of light components that reach an image sensing element via color filters differ depending on the filter colors, or the number of pixels of an image sensing element adopted per pixel of an output image differs for respective colors.
The gain correction for improving the sensitivity compensates for an insufficient amount of light by amplifying signal information of either an analog signal obtained from an image sensing element or a digital signal after A/D conversion, so as to allow a photographing operation even when the amount of light in a photographing environment is insufficient.
The saturation correction corrects, e.g., the saturation of blue to vividly express the color of clear sky, so as to obtain a preferred image by making colors in an image simulate colors in one's memory.
In the conventional method, flatness in an image is detected, and a noise removal process with a relatively high effect is applied to a flat portion. On the other hand, a noise removal process with a relatively low effect is applied to an edge portion or a process is skipped so as to minimize adverse effects.
However, as described above, the nature of noise superposed on an image signal does not depend on the flatness of an image. For this reason, with the conventional method, when a noise removal process that can sufficiently remove noise is applied to a given region, other regions suffer high noise level, and noise cannot be sufficiently removed. Furthermore, the adverse effects of the noise removal process may be visually conspicuous in other regions.
The present invention has been made in consideration of the aforementioned problems, and has as its object to provide an image processing apparatus and method, which can execute a noise removal process more effectively.